Response shift, recall bias and their effect on measuring change in health-related quality of life amongst older hospital patients

Background: Assessments of change in subjective patient reported outcomes such as health-related quality of life (HRQoL) are a key component of many clinical and research evaluations. However, conventional longitudinal evaluation of change may not agree with patient perceived change if patients' understanding of the subjective construct under evaluation changes over time (response shift) or if patients' have inaccurate recollection (recall bias). This study examined whether older adults' perception of change is in agreement with conventional longitudinal evaluation of change in their HRQoL over the duration of their hospital stay. It also investigated this level of agreement after adjusting patient perceived change for recall bias that patients may have experienced. Methods: A prospective longitudinal cohort design nested within a larger randomised controlled trial was implemented. 103 hospitalised older adults participated in this investigation at a tertiary hospital facility. The EQ-5D utility and Visual Analogue Scale (VAS) scores were used to evaluate HRQoL. Participants completed EQ-5D reports as soon as they were medically stable (within three days of admission) then again immediately prior to discharge. Three methods of change score calculation were used (conventional change, patient perceived change and patient perceived change adjusted for recall bias). Agreement was primarily investigated using intraclass correlation coefficients (ICC) and limits of agreement. Results: Overall 101 (98%) participants completed both admission and discharge assessments. The mean (SD) age was 73.3 (11.2). The median (IQR) length of stay was 38 (20-60) days. For agreement between conventional longitudinal change and patient perceived change: ICCs were 0.34 and 0.40 for EQ-5D utility and VAS respectively. For agreement between conventional longitudinal change and patient perceived change adjusted for recall bias: ICCs were 0.98 and 0.90 respectively. Discrepancy between conventional longitudinal change and patient perceived change was considered clinically meaningful for 84 (83.2%) of participants, after adjusting for recall bias this reduced to 8 (7.9%). Conclusions: Agreement between conventional change and patient perceived change was not strong. A large proportion of this disagreement could be attributed to recall bias. To overcome the invalidating effect of response shift (on conventional change) and recall bias (on patient perceived change) a method of adjusting patient perceived change for recall bias has been described.

Response shift in the assessment of quality of life among people attending cardiac rehabilitation

Objectives: To examine whether any response shift in quality of life assessment over the course of a cardiac rehabilitation programme could be explained by changes in individuals’ internal standards (recalibration), values (reprioritization) and/or conceptualization of quality of life and the extent to which any response shift could be explained by health locus of control, optimism and coping strategy. Design: Longitudinal survey design. Methods: The SEIQoL-DW was administered at the beginning and end of a cardiac rehabilitation programme. At the end of the programme, the SEIQoL-DW then-test was also administered to measure response shift. A total of 57 participants completed these measures and other measures to assess health locus of control, optimism and coping. Results: Response shift effects were observed in this population mainly due to recalibration. When response shift was incorporated into the analysis of QoL a larger treatment effect was observed. Active coping as a mechanism in the response shift model was found to have a significant positive correlation with response shift. Conclusion: This study showed that response shift occurs during cardiac rehabilitation. The occurrence of response shift in QoL ratings over time for this population could have implications for the estimation of the effectiveness of the intervention.

Response Shift and Glycemic Control in Children With Diabetes.

BACKGROUND: The purpose of this study was to investigate the scale recalibration construct of response shift and its relationship to glycemic control in children with diabetes. METHODS: At year 1, thirty-eight children with type 1 diabetes attending a diabetes summer camp participated. At baseline and post-camp they completed the Problem Areas in Diabetes (PAID) questionnaire. Post-camp, the PAID was also completed using the 'thentest' method, which requires a retrospective judgment about their baseline functioning. At year 2, fifteen of the original participants reported their HbA1c. RESULTS: PAID scores significantly decreased from baseline to post-camp. An even larger difference was found between thentest and post-camp scores, suggesting scale recalibration. There was a significant positive correlation between year 1 HbA1c and thentest scores. Partial correlation analysis between PAID thentest scores and year 2 HbA1c, controlling for year 1 HbA1c, showed that higher PAID thentest scores were associated with higher year 2 HbA1c. CONCLUSION: Results from this small sample suggest that children with diabetes do show scale recalibration, and that it may be related to glycemic control.

On the validity of measuring change over time in routine clinical assessment: a close examination of item-level response shifts in psychosomatic inpatients

Objective: Significant life events such as severe health status changes or intensive medical treatment often trigger response shifts in individuals that may hamper the comparison of measurements over time. Drawing from the Oort model, this study aims at detecting response shift at the item level in psychosomatic inpatients and evaluating its impact on the validity of comparing repeated measurements. Study design and setting: Complete pretest and posttest data were available from 1188 patients who had filled out the ICD-10 Symptom Rating (ISR) scale at admission and discharge, on average 24 days after intake. Reconceptualization, reprioritization, and recalibration response shifts were explored applying tests of measurement invariance. In the item-level approach, all model parameters were constrained to be equal between pretest and posttest. If non-invariance was detected, these were linked to the different types of response shift. Results: When constraining across-occasion model parameters, model fit worsened as indicated by a significant Satorra–Bentler Chi-square difference test suggesting potential presence of response shifts. A close examination revealed presence of two types of response shift, i.e., (non)uniform recalibration and both higher- and lower-level reconceptualization response shifts leading to four model adjustments. Conclusions: Our analyses suggest that psychosomatic inpatients experienced some response shifts during their hospital stay. According to the hierarchy of measurement invariance, however, only one of the detected non-invariances is critical for unbiased mean comparisons over time, which did not have a substantial impact on estimating change. Hence, the use of the ISR can be recommended for outcomes assessment in clinical routine, as change score estimates do not seem hampered by response shift effects.

Reference bias : presentation of extreme health states prior to EQ-VAS improves health-related quality of life scores. A randomised crossover trial.

Background: Clinical practice and clinical research has made a concerted effort to move beyond the use of clinical indicators alone and embrace patient focused care through the use of patient reported outcomes such as healthrelated quality of life. However, unless patients give consistent consideration to the health states that give meaning to measurement scales used to evaluate these constructs, longitudinal comparison of these measures may be invalid. This study aimed to investigate whether patients give consideration to a standard health state rating scale (EQ-VAS) and whether consideration of good and poor health state descriptors immediately changes their selfreport. Methods: A randomised crossover trial was implemented amongst hospitalised older adults (n = 151). Patients were asked to consider descriptions of extremely good (Description-A) and poor (Description-B) health states. The EQ-VAS was administered as a self-report at baseline, after the first descriptors (A or B), then again after the remaining descriptors (B or A respectively). At baseline patients were also asked if they had considered either EQVAS anchors. Results: Overall 106/151 (70%) participants changed their self-evaluation by ≥5 points on the 100 point VAS, with a mean (SD) change of +4.5 (12) points (p < 0.001). A total of 74/151 (49%) participants did not consider the best health VAS anchor, of the 77 who did 59 (77%) thought the good health descriptors were more extreme (better) then they had previously considered. Similarly 85/151 (66%) participants did not consider the worst health anchor of the 66 who did 63 (95%) thought the poor health descriptors were more extreme (worse) then they had previously considered. Conclusions: Health state self-reports may not be well considered. An immediate significant shift in response can be elicited by exposure to a mere description of an extreme health state despite no actual change in underlying health state occurring. Caution should be exercised in research and clinical settings when interpreting subjective patient reported outcomes that are dependent on brief anchors for meaning. Trial Registration: Australian and New Zealand Clinical Trials Registry (#ACTRN12607000606482) http://www.anzctr. org.au

Evidence of disagreement between patient-perceived change and conventional longitudinal evaluation of change in health-related quality of life among older adults

Objective: To identify agreement levels between conventional longitudinal evaluation of change (post–pre) and patient-perceived change (post–then test) in health-related quality of life. Design: A prospective cohort investigation with two assessment points (baseline and six-month follow-up) was implemented. Setting: Community rehabilitation setting. Subjects: Frail older adults accessing community-based rehabilitation services. Intervention: Nil as part of this investigation. Main measures: Conventional longitudinal change in health-related quality of life was considered the difference between standard EQ-5D assessments completed at baseline and follow-up. To evaluate patient-perceived change a ‘then test’ was also completed at the follow-up assessment. This required participants to report (from their current perspective) how they believe their health-related quality of life was at baseline (using the EQ-5D). Patient-perceived change was considered the difference between ‘then test’ and standard follow-up EQ-5D assessments. Results: The mean (SD) age of participants was 78.8 (7.3). Of the 70 participants 62 (89%) of data sets were complete and included in analysis. Agreement between conventional (post–pre) and patient-perceived (post–then test) change was low to moderate (EQ-5D utility intraclass correlation coefficient (ICC)¼0.41, EQ-5D visual analogue scale (VAS) ICC¼0.21). Neither approach inferred greater change than the other (utility P¼0.925, VAS P¼0.506). Mean (95% confidence interval (CI)) conventional change in EQ-5D utility and VAS were 0.140 (0.045,0.236) and 8.8 (3.3,14.3) respectively, while patient-perceived change was 0.147 (0.055,0.238) and 6.4 (1.7,11.1) respectively. Conclusions: Substantial disagreement exists between conventional longitudinal evaluation of change in health-related quality of life and patient-perceived change in health-related quality of life (as measured using a then test) within individuals.

Differences in the symptom experience of older oncology outpatients

Context The relatively low number of older patients in cancer trials limits knowledge of how older adults experience symptoms associated with cancer and its treatment. Objectives This study evaluated for differences in the symptom experience across four older age groups (60–64, 65–69, 70–74, ≥75 years). Methods Demographic, clinical, and symptom data from 330 patients aged >60 years who participated in one Australian and two U.S. studies were evaluated. The Memorial Symptom Assessment Scale was used to evaluate the occurrence, severity, frequency, and distress of 32 symptoms commonly associated with cancer and its treatment. Results On average, regardless of the age group, patients reported 10 concurrent symptoms. The most prevalent symptoms were physical in nature. Worrying was the most common psychological symptom. For 28 (87.5%) of the 32 Memorial Symptom Assessment Scale symptoms, no age-related differences were found in symptom occurrence rates. For symptom severity ratings, an age-related trend was found for difficulty swallowing. As age increased, severity of difficulty swallowing decreased. For symptom frequency, age-related trends were found for feeling irritable and diarrhea, with both decreasing in frequency as age increased. For symptom distress, age-related trends were found for lack of energy, shortness of breath, feeling bloated, and difficulty swallowing. As age increased, these symptoms received lower average distress ratings. Conclusion Additional research is warranted to examine how age differences in symptom experience are influenced by treatment differences, aging-related changes in biological or psychological processes, or age-related response shift.

Assessing quality of life in cardiac rehabilitation: Choosing an appropriate tool

Quality of life is an important outcome for people undergoing cardiac rehabilitation. This paper discusses the difficulties with defining the concept of quality of life and how it might be distinct from the concept of health-related quality of life. Based on a review of the literature, a description is provided of health-related quality of life questionnaires that have been used in cardiac rehabilitation populations. Some criteria for choosing between these questionnaires are then discussed and, finally, a brief discussion is presented of the concept of response shift and how this might influence the assessment of health-related quality of life in a cardiac rehabilitation setting.

Emotional well-being and adjustment to vision loss in later life: a meta-synthesis of qualitative studies.

Purpose: To review perceived emotional well-being in older people with visual impairment and perceived factors that inhibit/facilitate psychosocial adjustment to vision loss. Method: The databases of MEDLINE, EMBASE, PsycINFO and CINAHL were searched for studies published from January 1980 to December 2010, which recruited older people with irreversible vision loss, and used qualitative methods for both data collection and analysis. Results sections of the papers were synthesised using a thematic-style analysis to identify the emergent and dominant themes. Results: Seventeen qualitative papers were included in the review, and five main themes emerged from the synthesis: 1) the trauma of an ophthalmic diagnosis, 2) impact of vision loss on daily life, 3) negative impact of visual impairment on psychosocial well-being, 4) factors that inhibit social well-being, and 5) factors that facilitate psychological well-being. We found the response shift model useful for explaining our synthesis. Conclusions: Acquired visual impairment can have a significant impact on older people's well-being and make psychosocial adjustment to the condition a major challenge. Acceptance of the condition and a positive attitude facilitate successful psychosocial adjustment to vision loss as well as social support from family, friends and peers who have successfully adjusted to the condition. [Box: see text].

Change of perspective: a measurable and desired outcome of chronic disease self-management intervention programs that violates the premise of preintervention/postintervention assessment

Objective: To determine if participants in chronic disease self-management courses have a change of perspective of their health status (a response shift), and if this is measurable with a paper-based questionnaire.

Methods: Nine items were developed to measure potential benefits of self-management courses. These were based on the constructs of a previous questionnaire, the Health Education Impact Questionnaire (HEI-Q). Cognitive interviews elicited spontaneous statements about the reasons for paper-based answers. Sensitivity, specificity, and overall accuracy of items were calculated using the interview as a relative gold standard. Response shift can be negative (i.e., after the course, participants realize that, before the course, they were worse than they thought they were), positive (i.e., participants now realize they were better than they thought they were), or absent (no change).

Results: Interviews (n = 39) reflected that true response shift occurred in approximately half the replies to questionnaire items. Of these, 31% were negative response shift, 20% were positive response shift. Response shift was absent in 32% of replies. Presence or absence of response shift could not be determined in 17% of replies across items. Significant concordance between questionnaires and cognitive interviews (average overall accuracy 0.79) indicated that the HEI-Q Perspective questionnaire detects response shift in participants of self-management courses. The questionnaire revealed that 87% of participants had response shift in at least 1 item.

Conclusion: This study suggests that preintervention/postintervention assessments of interventions such as self-management courses are confounded by a change in perspective of a large proportion of respondents. It also indicates response shift is a valuable outcome of self-management courses that can be measured with a paper-based questionnaire.